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ABSTRACT 



The General Accounting Office (GAO) reviewed the labor 
market links and vocational training performance of the Job Corps, which is 
an employment and training program aimed at providing severely disadvantaged 
youth with a comprehensive array of services in residential settings. Data 
were collected from the following: interviews with Job Corps officials at the 
regional and national levels; site visits of six " judgmental ly" selected Job 
Corps centers (five established centers and one recently opened center) ; and 
data from the Department of Labor's Student Pay, Allotment, and Management 
Information System. The review established that the Department of Labor is 
working to ensure the appropriateness of its vocational training to local 
labor markets through the following activities: industry advisory groups, a 
school-to-work initiative linking Job Corps with local employers, and 
involvement of local business and community leaders in deciding which 
vocational training programs to offer at new Job Corps centers. However, Job 
Corps -supplied data on program completion rates and participants' ability to 
obtain training-related jobs remain misleading and continue to overstate 
program results. (Thirteen tables/figures are included. Appended are the 
following: notes on scope and methodology, data supporting report figures, 
comments- from Department of Labor, GAO contacts and staff acknowledgments, 
and related GAO products.) (MN) 
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United States 

General Accounting Office 

Washington, D.C. 20548 



Health, Education, and 
Human Services Division 

B-279218 

November 4, 1998 

The Honorable Christopher Shays 
Chairman, Subcommittee on Human Resources 
Committee on Government Reform 
and Oversight 
House of Representatives 

Dear Mr. Chairman: 

Job Corps is an employment and training program aimed at providing 
severely disadvantaged youths with a comprehensive array of services, 
generally in a residential setting. The Department of Labor spends about 
$1 billion under this program each year to serve more than 68,000 youths, 
who, on average, spend about 7 months in the program. On average, the 
cost per participant is more than $15,000, making Job Corps the nation’s 
most expensive job training program. One of the services that is critical to 
the program’s success is vocational training. Each of Job Corps’ 113 
centers offers training in several vocational areas, such as automotive 
trades, health occupations, construction trades, culinary arts, clerical 
occupations, and building and apartment maintenance. Vocational training 
services can be provided by Job Corps center staff, private providers 
under contract to the Job Corps center, or national labor and business 
organizations under contract with Job Corps’ national office. 

In the past, we have assessed various aspects of the Job Corps 
program — outreach, admissions, and placement; costs and results; 
similarities to state youth training initiatives; and the extent to which the 
program is locally based. 1 Reflecting your continued interest in the Job 
Corps program, you requested that we examine Job Corps’ vocational 
training component to describe the program’s contracting policies and 
procedures and to assess contractor performance. Specifically, you asked 
us to determine how Job Corps ensures that vocational training is 
appropriate and relevant to employers’ needs and the extent to which 
participants are completing vocational training and obtaining 
training-related jobs. In addition, you asked us to describe Job Corps’ 



' Job Corps: Need for Better Enrollment Guidance and Improved Placement Measures 
(GAO/HEHS-98-1, Oct. 21, 1997); Job Corps: High Costs and Mixed Results Raise Questions About 
Program’s Effectiveness (GAO/HEHS-95-180, June 30, 1995); Job Corps: Comparison of Federal 
Program With State Youth Training Initiatives (GAO/HEHS-96-92, Mar. 28, 1996); and Job Corps: Where 
Participants Are Recruited, Trained, and Placed in Jobs (GAO/HEHS-96-140, July 17, 1996). 
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Results in Brief 



o 
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process for contracting with vocational training providers, in particular 
Labor’s use of sole source contracting with national training contractors. 2 

In carrying out our work, we met with Labor officials and reviewed 
Labor’s policies and procedures for contracting for vocational services. 

We obtained data on program outcomes for Job Corps participants during 
the period July 1, 1995, to June 30, 1997, the two most recently completed 
program years. 3 We visited five centers that used a variety of methods to 
provide vocational training and that served a majority of local 
participants — those who resided within 100 miles of a center. At these 
centers, we reviewed participant data on vocational completion status and 
placement information. In reviewing job training match information, we 
used reported placement information and the Dictionary of Occupational 
Titles and called a number of employers to clarify reported information. 
We visited a sixth center — one that recently opened — to gather 
information on its experience in deciding which vocational training 
courses to offer. We performed our work between January and July 1998 
in accordance with generally accepted government auditing standards. 
(App. I contains a more detailed discussion of our scope and 
methodology.) 



Labor has several activities to foster Job Corps’ employer and community 
linkages to ensure the appropriateness of its vocational training to local 
labor markets and its relevance to employers’ needs. Labor has industry 
advisory groups that regularly review vocational course curricula to 
ensure their relevance to today’s job market. Labor has also introduced a 
school-to-work initiative designed to link Job Corps with local employers, 
combining center-based training with actual worksite experience at more 
than half the Job Corps centers. In addition, Labor involves local business 
and community leaders in deciding which vocational training programs to 
offer at newly established Job Corps centers. Complementing these 
national efforts, three of Labor’s regional offices have developed their own 
initiatives to improve linkages between Job Corps and local labor markets, 
including modifying existing vocational offerings to meet local employer 
needs. 



-We presented preliminary findings on this work in testimony before the Human Resources 
Subcommittee of the House Committee on Government Reform and Oversight: Job Corps: Vocational 
Training Performance Data Overstate Program Success (GAO/T-HEHS-98-218, July 29, 1998). 

’A program year begins on July 1 of a year and ends on June 30 of the following year. A program year is 
designated by the year in which it begins. Thus, program year 1996 began on July 1, 1996, and ended on 
June 30, 1997. 
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Despite Labor’s efforts to increase the effectiveness of its vocational 
training through employer and community linkages, Job Corps data on the 
extent to which participants complete vocational training and obtain 
training-related jobs are misleading and overstate the program’s results. 
Although Job Corps reported that 48 percent of its program year 1996 
participants completed their vocational training, we found that only 
14 percent of the program participants actually completed all the 
requirements of their vocational training curricula. The rest of the 
participants whom Job Corps considered to be vocational completers had 
performed only some of the duties and tasks of a specific vocational 
training program. Labor also reported that 62 percent of the participants 
nationwide who obtained employment found jobs that matched the 
vocational training received in Job Corps. At the five centers we visited, 
however, the validity of about 41 percent of the job placements reported 
by Labor to be training-related was questionable. Examples of 
questionable training-related job placements include several participants 
at one center receiving clerical training but reported by Labor as getting 
jobs as bank tellers at fast food restaurants, retail stores, and a gas station; 
several participants at another center were trained in health occupations 
and were reported as getting jobs as information clerks at various 
restaurants and a car rental agency. 

Finally, in looking at how training providers are selected, we found that 
about a third of Job Corps’ vocational training has been provided under 
sole source contracts awarded to national labor and business 
organizations for more than 30 years, but in our opinion, Labor has not 
adequately justified procuring these training services noncompetitively. A 
principal reason Labor has cited for awarding these contracts on a sole 
source basis is that these organizations maintain an extensive nationwide 
placement network and are better able than nonnational organizations to 
place Job Corps participants who complete their training. Labor has 
provided no data, however, to show the extent to which these sole source 
contractors actually place Job Corps participants nationwide. 

In light of our work, we are recommending to the Secretary of Labor 
actions to more accurately assess Job Corps’ accomplishments and 
adequately justify the use of sole source contracts for vocational training. 



Background 



Job Corps was established as a national employment and training program 
in 1964 to address employment barriers faced by severely disadvantaged 
youths. Job Corps enrolls youths aged 16 to 24 who are economically 
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disadvantaged, in need of additional education or training, and living 
under disorienting conditions such as a disruptive homelife. 4 In program 
year 1996, nearly 80 percent of the participants were high school dropouts 
and almost two-thirds had never been employed full-time. Participating in 
Job Corps can lead to placement in a job or enrollment in further training 
or education. It can also lead to educational achievements such as 
attaining a high school diploma and improving reading or mathematics 
skills. 

Job Corps currently operates 113 centers throughout the United States, 
including Alaska, Hawaii, the District of Columbia, and Puerto Rico. Major 
corporations and nonprofit organizations manage and operate 85 Job 
Corps centers under contractual agreements with Labor. Contract center 
operators are selected through a competitive procurement process that 
takes into account proposed costs, an operator’s expertise, and prior 
program performance. In addition, the U.S. Department of the Interior and 
the U.S. Department of Agriculture operate 28 Job Corps centers, called 
civilian conservation centers, on public lands under interagency 
agreements with Labor. Each center provides participants with a wide 
range of services, including basic education, vocational skills training, 
social skills instruction, counseling, health care, room and board, and 
recreational activities. 

One feature that makes Job Corps unique is that, for the most part, it is a 
residential program. About 90 percent of the youths enrolled each year live 
at Job Corps centers and are provided services 24 hours a day, 7 days a 
week. The premise for boarding participants is that most come from a 
disruptive environment and, therefore, can benefit from receiving 
education and training in a different setting where a variety of support 
services is available around the clock. The comprehensive services Job 
Corps provides make it a relatively expensive program. According to 
Labor’s program year 1996 figures, the average cost per Job Corps 
participant was more than $15,000. Cost varies according to how long Job 
Corps participants remain in the program. Participants stay in the program 
for an average of about 7 months but may stay as long as 2 years. Labor 
estimates the cost for a participant who remains in the program for a year 
to be about $25,000. 

Vocational training is a critical element of the Job Corps program. This 
training is designed to offer individualized, self-paced, and open 



4 Although the act includes 14- and 15-year-old youths in the age criteria, Job Corps regulations provide 
that youths 14 and 15 years of age may be eligible “upon a specific determination by the program 
director to enroll them.” 
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entry-open exit instruction to allow participants to progress at their own 
pace. Vocational training can be provided in any combination of three 
ways. Most vocational training is offered by instructors who are Job Corps 
center staff. Other vocational courses are taught by private providers 
under contract to the center. These private providers typically include 
vocational schools and community colleges. About a third of the 
vocational training expenditure is provided by national labor unions and 
business organizations under sole source contracts with Labor. In program 
year 1996, Job Corps’ operating costs totaled about $986 milhon, of which 
$144 million, or about 15 percent, was for vocational training (see table 1). 



Table 1 : Job Corps Operating Costs 
for Program Year 1996 


Expense category 


Amount (millions) 


Percentage of 
total 




Student training costs 


$424.7 


43.1 




Basic education 


72.6 


7.4 




Vocational training 


144.0 


14.6 




Social skills training 


208.1 


21.1 




Supportive services 


378.0 


38.3 




Outreach and intake 


41.4 


4.2 




Transportation 


14.5 


1.5 




Meals and lodging 


146.6 


14.8 




Allowances 


81.6 


8.3 




Workers compensation benefits 


4.5 


0.5 




Medical care 


56.4 


5.7 




Placement and settlement 


32.9 


3.3 




Administration 


183.1 


18.6 




Total 


$985.8 


100.0 



Source: Job Corps Annual Report, Program Year 1996, Department of Labor, Employment and 
Training Administration. 



Overall, Job Corps offers training in 100 different vocations. Although the 
number of vocations offered at any one Job Corps center varies, most 
centers offer training in 7 to 10 different vocations. Some centers, 
however, offer training in as few as 5 vocations while others offer training 
in as many as 31 different vocations. Some vocations are available at most 
centers, while others are available at only a single center. For example, 
more than 80 percent of the centers offer training in business clerical, 
culinary arts, building and apartment maintenance, and carpentry. 
Thirty-one vocations, including computer programmer, asphalt paving, 
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barber, teacher aide, and cable TV installer, are offered only at a single 
center. 

Many centers also offer off-site advanced career training at such 
institutions as vocational schools, community colleges, and universities for 
participants who have been in the program for at least 6 months. 
Regardless of who provides the training, Job Corps policy requires that all 
vocational training programs use competency-based curricula that contain 
a series of skills, or competencies, that participants must attain. According 
to Labor officials, each vocational training program’s curriculum and set of 
required skills are regularly reviewed and updated by industry advisory 
groups consisting of business, industry, and training providers. 

Labor uses a series of nine measures to report on the performance of the 
program nationally and to assess the performance of individual Job Corps 
centers. The measures relate to placement — in a job, in education, or in 
military service — learning gains in mathematics and reading, earning a 
general equivalency diploma certificate, completing vocational training, 
placement in a job related to the training received, and placement wage. In 
program year 1996, Job Corps reported that 80 percent of the participants 
leaving the program were placed — 70 percent in jobs or the military and 
10 percent enrolled in education — and 62 percent of those who were 
placed in jobs or the military obtained a job related to their training. Job 
Corps also reported that 48 percent of those who left the program 
completed vocational training. 



Labor has several activities to improve Job Corps’ employer and 
community linkages to ensure that vocational training is appropriate for 
local labor markets and relevant to employers’ needs. These efforts 
include initiatives enacted by Job Corps’ national office and regional 
offices, as well as efforts by individual Job Corps centers. 



National Initiatives Since 1984, Labor has used industry advisory groups to review vocational 

Recognize Need for course curricula to ensure that course content is relevant to the job 

Employer Involvement market. Each year, Labor selects a number of vocational offerings for 

review by an Industry Advisory Group consisting of Job Corps instructors 
and academic program representatives as well as industry representatives 
from each vocational offering being reviewed. For example, recent 
industry representatives included computer operators and repair 



Labor’s Initiatives 
Enhance the 
Appropriateness and 
Relevancy of 
Vocational Training 
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technicians, electronic assemblers, diesel and heavy equipment 
mechanics, health occupation workers, material handlers, tile setters, and 
clerical workers. The Industry Advisory Group recommends to Labor 
changes to Job Corps’ vocational training curricula, materials, and 
equipment. Vocational offerings are evaluated and updated on a 3-to-5-year 
cycle dictated by industry changes and the number of students 
participating in each vocational training program. 

In program year 1995, Labor introduced a school-to-work initiative at three 
Job Corps centers combining center-based training with actual worksite 
experience related to it. Labor expanded this initiative to an additional 30 
centers in program year 1996 and to 30 more centers in program year 1997. 
Labor provided financial incentives and supportive services to encourage 
centers to participate in the school-to-work initiative. According to Labor 
officials, the school-to-work initiatives have resulted in extensive 
partnerships being established between the centers, area businesses, and 
local school systems. Through these partnerships, employers are providing 
worksite learning experiences, suggesting approaches for integrating 
curricula, developing assessment criteria for documenting skill mastery, 
and participating in career exposure activities. At one school-to-work Job 
Corps center that we visited, 35 participants from program year 1996 were 
involved in this initiative and all were placed — 32 had jobs, 2 returned to 
school, and 1 joined the military. Furthermore, 70 percent of the jobs were 
directly related to the vocational training received in Job Corps. 

Labor also involves local business and community leaders in deciding 
which vocational training programs are to be offered at newly established 
Job Corps centers. For example, at the new center we visited, we found 
that 2 years prior to the awarding of the center’s contract, decisions on the 
vocations to be offered were made with input from local business and 
community leaders, including representatives of the mayor’s office, the 
private industry council, the school department, and local businesses. 5 The 
result was that this center does not offer many of the traditional Job Corps 
vocational programs, such as clerical, culinary arts, landscaping, and 
building and apartment maintenance. Instead, it has nine vocational areas 
in such high-demand occupations as medical assistant, phlebotomy and 
EKG technician, and computer repair. At another new center, Labor 
officials stated that local labor market information along with input from 



5 Under the Job Training Partnership Act of 1983, a private industry council, comprising representatives 
of private sector employers, local education agencies, organized labor, rehabilitation agencies, 
community-based organizations, economic development agencies, and the public service employment 
agency, is appointed by local elected officials of each service delivery area and approves a job training 
plan designed to meet local employment and training needs. 
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local community and business leaders, including the local private industry 
council, union representatives, local school system, health groups, and 
chamber of commerce, ensured that the vocational training courses 
offered at that center would be appropriate and current given the local 
economy. 

Labor officials also informed us that changes to vocational training 
offerings at existing centers result from changes in labor market demand 
or poor performance of a particular vocational training program. Centers 
obtain approval for a change by completing the appropriate paperwork for 
a request for change and submitting it to either the regional office (if the 
change involves a center-operated or center-contracted vocational 
offering) or the national office (if the change involves a vocational course 
offered by a national labor union or business organization). Labor then 
assesses the request to change course offerings and reviews the placement 
analyses, wages reported, female participation rate in the course, local 
labor market information, and facility requirements. In addition, Labor 
requires the center to obtain statements from three employers stating that 
the vocational change is appropriate and relevant. All five of the centers 
we visited had recently made changes to their vocational course offerings. 
For example, one center added a physical therapy course after receiving 
numerous requests from clinics and hospitals within the community. The 
center was able to add this course by dropping a cosmetology course. 
Another center identified a local demand for qualified workers in retail 
sales and tourism. The center added training in these vocations while 
reducing the size of its clerical training program. 



Regional Job Corps 
Initiatives Expand 
Employer Relations 



In addition to national efforts, three of Labor’s regional offices have 
developed their own initiatives to improve linkages between Job Corps 
centers and employers. In one region, business leaders representing a 
variety of industries met with Labor and center staff to provide 
observations of the program and the participants they hire. The group — a 
business roundtable — set up a framework for obtaining employer input 
into the operation of the Job Corps program for the benefit of young 
people, employers, community leaders, and the Job Corps system 
nationwide. In an effort to bridge the gap between the needs of private 
industry and vocational training, the roundtable recommended actions and 
supported the implementation of new strategies to resolve employer 
issues that it identified and prioritized. As a direct result of this 
roundtable, concrete linkages were established. For example, a bank 
involved as a school-to-work program participant provided equipment and 
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instructors to incorporate bank telling into the center’s clerical program. 
According to Labor officials, the initiative was successful, and the regional 
office is currently exploring the possibility of duplicating this effort in 
several other Job Corps centers. At another center within the region, an 
electronics firm reviewed the center’s electronics curriculum and 
suggested additional skills allowing program participants to qualify for 
higher-paying jobs. 

Another region has endorsed a major initiative between a Job Corps center 
and the Q-Lube Corporation whereby a building at the center was 
renovated to exactly meet the specifications of a Q-Lube facility. The 
renovation used student painters and carpenters from other vocational 
training courses and Job Corps provided additional funding for this 
course. Q-Lube donated the equipment to the center and also provided a 
trained instructor. The course offering is identical to the program 
curriculum Q-Lube teaches at non-Job Corps sites. According to Labor 
officials, since the implementation of this initiative, Q-Lube has become a 
major employer and training link within the region. 

The same regional office contacted a shipbuilding company advertising for 
500 shipbuilders and worked with the company to develop a vocational 
training program in welding for Job Corps students that would be 
appropriate and relevant to the company’s needs. The company provided 
the two pieces of equipment needed for training purposes. Students were 
trained at the Job Corps center under conditions similar to those in the 
shipbuilding environment, tested by the company, and then provided 
additional training at the shipbuilding site. In addition, the company 
provided low-cost housing and full salary to students who passed the test 
before graduating from the center. The company was pleased with the 
students’ qualifications, attitudes, and work ethics and requested that the 
Job Corps program train another 100 students. The region is currently 
recruiting and training students for this vocation in an attempt to further 
meet the needs of the shipbuilding industry. 

A third regional office is involved in a project to increase the involvement 
of employers in all facets of Job Corps operations in their region, including 
curriculum development, customized training, work-based learning, 
mentoring, identifying workforce needs, and donating staff resources and 
equipment. The goal of this outreach campaign is to build substantial 
relationships between Job Corps and the employer community at several 
different but mutually supportive levels: center, state, regional, and 
national. Labor selected a contractor through a competitive process, 
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assisted by several national groups, to research, test, and revise its 
proposed strategy for increasing employer involvement within the region. 
Initially, the project concentrated on three centers in different states 
within the region. The project will soon expand to include all states and 
Job Corps centers within the region. If successful, the project will be 
expanded throughout the Job Corps system. 



Job Corps centers have also independently established linkages with 
employers. These linkages include negotiating with employers to provide 
furniture and vocational training equipment and contracting with 
employers to train and hire program participants. For example, at one 
center a national employer has donated computers, copy machines, desks, 
chairs, and conference tables valued at approximately $50,000. At another 
center, an automobile maker has donated a four-wheel-drive sport utility 
vehicle for students in the auto repair vocational training course in an 
attempt to make the training more relevant to the vehicles that students 
would actually be working on. The center is currently working with the 
automobile maker to donate a car for the same purpose. Local automobile 
dealers are familiar with the center’s linkages to the national automobile 
maker and also have donated cars needing repair. In addition, local 
automobile dealers have trained students through the school-to-work 
program and have hired many of the Job Corps program participants. 

Another center holds monthly employer relations meetings in which 
approximately 200 local employers and community representatives attend 
a luncheon catered by the center’s culinary arts students. Speakers discuss 
local employment opportunities and donate funds to benefit Job Corps 
participants. The funds, which are managed by the center’s Community 
Relations Council, sure used to provide tuition scholarships for program 
graduates continuing their education upon completion from the center. 

The scholarships range from $500 to $1,000 each and are awarded to 
program graduates who have pursued excellence and attained a higher 
measure of success than their fellow program participants. To date, about 
$10,000 has been raised for scholarships. 

A fourth center established an effective business relationship with a 
computer graphics firm in California. According to center officials, 31 Job 
Corps students enrolled in various vocational training programs, including 
building and apartment maintenance, clerical, electrical, and landscaping; 
participated in 12-week internships at the computer firm; and attended an 
anger management course that had been developed for the firm’s 



Local Job Corps Centers 
Develop Their Own 
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employees. These students earned $10 per hour within a work-based 
environment in which the firm’s staff provided on-the-job training and 
mentoring. The center placement official claims that the success of the 
internship program is evidenced by the 28 students who obtained 
primarily training-related jobs after terminating from the Job Corps 
program. 



Two Job Corps 
Program Measures 
Are Misleading and 
Overstate Program 
Success 



Two performance indicators that Labor uses to evaluate Job Corps’ 
success are misleading, overstating the extent to which vocational training 
is completed and job placements are training-related. Labor reports that 
nationwide about 48 percent of all program participants complete their 
vocational training and that about 62 percent of the jobs obtained by 
program participants are related to the training they received. However, 
we found that nationally only about 14 percent of the program participants 
satisfied all their vocational training requirements and that about 
41 percent of the reported training-related job placements at the five 
centers we visited were questionable. Having complete and accurate 
program performance information is important to evaluating program 
success and being able to identify areas needing improvement. 6 



Vocational Completion 
Does Not Mean Vocational 
Training Was Completed 



Nationally, Job Corps reported that in program year 1996, 48 percent of its 
participants completed vocational training. This information is misleading. 
We found that only about 14 percent of the program year 1996 participants 
actually completed all the required tasks of their vocational training 
programs. Job Corps’ national data system uses three categories to 
identify a participant’s level of vocational training progress: trainee, 
completer, and advanced completer. A trainee is a participant who has not 
completed any vocational training component, a completer has 
accomplished at least one component of a vocational program, and an 
advanced completer has fully satisfied all required components of a 
vocational training program. Labor considers participants in the last two 
categories to be vocational training completers. Thus, Job Corps 
vocational completion statistics include participants who have only 
partially completed the required skills of a vocational training program. 7 



'In GAO/HEHS-95-180, June 30, 1995, we also questioned the validity of 15 percent of the reported 
placements we sampled at six centers. 

'Placement in a job is not dependent on a participant’s completing any of the required skills of a 
vocational training program. Job Corps requires placement contractors to assist all participants with 
placement, regardless of how long they were in the program or the reason they left. 
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Each Job Corps vocational training program has a comprehensive list of 
duties and tasks that participants are expected to perform. For example, 
the clerical vocational training program has 140 duties and tasks that must 
be mastered to fully complete the program, food service has 109, building 
and apartment maintenance has 123, and carpentry has 75. Vocational 
training programs, however, can be divided into several components. For 
example, in food service, the first component entails making a sandwich 
and preparing a salad (covering 39 of the 109 tasks). The second 
component adds preparing breakfast dishes; heating convenience foods; 
preparing meats, poultry, fish, and pasta; and cooking vegetables. The final 
component adds preparing soups, sauces, and appetizers as well as food 
management skills, such as preparing a menu, setting a table, developing a 
food preparation schedule, and conducting safety inspections. 

Vocational training instructors assess participants’ performance for each 
duty and task, and Job Corps policy permits participants to be classified as 
vocational completers if they accomplish the duties and tasks associated 
with any one component of the vocational training program — regardless of 
whether they can perform all the duties and tasks required in the entire 
vocational training curriculum. Depending on the vocation, the percentage 
of tasks that a participant must accomplish to be considered a completer 
range from virtually all, as in the health occupations program, to about a 
quarter, as in the welding program (see table 2). 



Table 2: Tasks Participants Must 
Accomplish to Be Considered 




Number of tasks 


Minimum number 




Vocational Completers in Selected 




to fully complete 


of tasks required 


Percentage of 


Vocational Training Programs 


Vocation 


training 


of a completer 


total tasks 



Health occupations 


189 


179 


95 


Electrical 


79 


56 


71 


Painting 


50 


32 


64 


Auto repair 


59 


36 


61 


Carpentry 


75 


42 


56 


Building and apartment 
maintenance 


123 


64 


52 


Clerical 


140 


67 


48 


Landscaping 


167 


71 


43 


Bricklaying 


64 


26 


41 


Food service 


109 


39 


36 


Welding 


128 


36 


28 
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Thus, Job Corps policy allows participants to be classified as vocational 
completers if they can perform some portion of a required curriculum. For 
example, in the food service vocational training program, accomplishing 
just the tasks associated with the salad and sandwich making component 
would qualify a participant as a vocational completer. At the centers that 
we visited that had a food service program, nearly half of the reported 
vocational completers had completed only this first component. Similarly, 
nearly 80 percent of the vocational completers in the carpentry program at 
five centers completed only the first of three components. In contrast, 
about 15 percent of the vocational completers of the centers’ health 
occupations program completed only the first of two components (see fig. 
1). Overall at the five centers, 43 percent of the vocational completers 
completed only the first component of their vocational training programs. 



Figure 1: Vocational Completers Who Completed Only the First Component of Their Training for Selected Vocations at Five 
Centers 

Percentage of Vocational Completers 




and Apartment Occupations 

Maintenance 



Vocational Training Program 



The reported percentage of vocational completers at the five centers we 
visited substantially overstated the percentage of participants who fully 
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completed their vocational training programs. At these centers, about 
51 percent of the 3,500 participants were considered to be vocational 
completers. However, only about 18 percent completed all their vocational 
training requirements. As shown in figure 2, the percentage of program 
year participants fully completing vocational training programs ranged 
from about 11 percent at one center to about 27 percent at another center. 
Nonetheless, these two centers had reported vocational completion rates 
of 65 percent and 73 percent, respectively. 




Figure 2: Reported and Full Completion Rates for Vocational Training Nationwide and at Five Job Corps Centers for 
Program Year 1996 



Percentage of Participants 




Job Corps Center 

□ Partial Completers 
9 Full Completers 



Closer examination of the participants who completed only the first 
component of their vocational training program showed that many spent a 
short period of time — less than 90 days — enrolled in vocational training. At 
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the five centers that we visited, nearly 15 percent of the participants who 
had completed the first component of their vocational training spent fewer 
than 90 days in training. This ranged from about 9 percent at one center to 
about 20 percent at another center. 



Job Training Match 
Statistics Are Not 
Meaningful 



Labor reported that in program year 1996, 62 percent of participants 
placed in employment found jobs that matched the training they received 
in Job Corps. Our review of this information at the five centers we visited, 
however, suggests that this report substantially overstates the program’s 
accomplishments. We found that the validity of about 41 percent of the 
reported job training matches at these centers was questionable. 

In a previous report, we expressed concern with Labor’s methodology for 
identifying training-related placements. 8 We concluded that Labor gave its 
placement contractors wide latitude in deciding whether a job was a job 
training match and identified many jobs that appeared to bear little, if any, 
relationship to the training received. We also noted that placement 
contractors used some creativity when reporting job titles in order to 
obtain a job training match. Labor questioned the accuracy of claims made 
by placement contractors that job training matches could be obtained for 
participants trained as bank tellers, secretaries, and welders who obtained 
jobs in fast food restaurants. 

In checking reported job training match information, we reviewed all 
reported training-related job placements at the five centers we visited to 
assess the validity of reported job training matches. We verified the results 
by contacting a representative sample of employers who had hired the Job 
Corps participants. 9 In this process, we questioned a significant number of 
the claimed matches. We questioned job training matches because either a 
job title did not seem appropriate for the employer listed (such as bank 
teller at a fast food restaurant) or the job title did not seem to relate to the 
vocational training (such as a job as an information clerk at a car rental 
agency after training as a home health aide). We then interviewed a 
random sample of 183 employers who hired Job Corps participants whose 
job placement was listed as related to the vocational training they received 



8 GAO/HEHS-98-l , Oct. 21, 1997. 

!> We identified 598 questionable job training matches at the five centers. We contacted a random 
sample of 183 of the listed employers to verify information about the reported job placement and used 
the results of this sample to estimate the number of questionable matches. We estimate, at the 
95-percent confidence level, that between 519 and 563 job training matches are questionable at the five 
centers. 
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but that we questioned. 10 Table 3 shows additional questionable examples 
of jobs reported as being training-related. 



Table 3: Examples of Reported Job 
Training Matches We Questioned 


Vocational training 


Placement job 


Employer 




Accounting 


Bank teller 

Linen room attendant 


Cable Car Cleaners 
Sheraton Hotel 




Auto repair 


Detailer 
Wash boy 


Vintage Car Wash 
Mesa Ford 




Bricklaying 


Janitor 

Material handler 


Waffle House 
Roger’s Supermarket 




Carpentry 


Municipal maintenance 
worker 

Stone polisher 


Piggly Wiggly’s grocery 
store 

Ro-An Jewelers 




Clerical 


Bank teller 
Cashier 


McDonald’s 
Dunkin Donuts 




Diesel mechanic 


Laundry machine washer 
Stock checker 


Elks Lodge 
K-Mart 




Electronic assembly 


Car wash attendant 
Machine cleaner 


Vintage Car Wash 
Baskin-Robbins 




Food service 


Housekeeper 
Personal attendant 


Pilar De La Torre 
Consuelito’s Boutique 




Home health aide 


Appointment clerk 
Information clerk 


Vision Dry Cleaning 
Alamo Car Rental 




Hotel or motel clerk 


Fast food worker 
Ticket seller 


McDonald’s 
Regal Theaters 




Medical assistant 


Information clerk 
Sanitarian 


Delia's Restaurant 
Wendy’s 




Painting 


Janitor 

Material handler 


McDonald’s 
Federal Express 




Plumbing 


Assembler 
Material handler 


Sealy Mattress 
UPS 




Welding 


Material handler 
Utility worker 


Popeye's Chicken 
KC Pools 



At the five centers we visited, we questioned 598 of the 1,306 reported job 
training matches. The percentages of these questionable job training 
matches ranged from about 30 percent at one center to about 64 percent at 
another center (see fig. 3). 



u, We also question Job Corps’ policy of providing an automatic job training match for any individual 
enlisting in the military, regardless of the assigned duties. At the five centers, military enlistments 
accounted for about 5 percent of the reported job training matches, although at one center about 
13 percent of the reported job training matches were for military enlistments. However, we did not 
include these reported job training matches in our questionable category. 
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Figure 3: Reported and Questionable Training-Related Job Placement Rates at Five Job Corps Centers 
Percentage 
80 



60 
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I I Questionable 



Note: Reported rates are the full bar heights. 



Our discussions with employers yielded examples of jobs that, on the 
surface, were related to the training received, based on the reported job 
title, but were actually quite unrelated to this training. For example, one 
participant trained in welding was reported as obtaining a job as a welding 
machine operator at a temporary agency, but the employer informed us 
that this individual was actually hired to shuttle vehicles between airports. 
Another participant trained in auto repair was reportedly hired as a 
petroleum and gas laborer but was actually hired to clean residential 
homes. A third participant received clerical training and was reportedly 
hired as a sales correspondent but actually sorted bad tomatoes from good 
ones on a conveyor belt. All three of these Job Corps participants, 
therefore, were erroneously reported as having been placed in jobs related 
to their training. 
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Labor’s monitoring of reported job training matches appears to be 
inadequate. Labor officials stated that Job Corps’ regional offices are 
responsible for monitoring all aspects of placement contractor 
performance but that there is no fixed schedule for such monitoring. They 
stated that regular desk reviews of all placement forms, for both accuracy 
and completeness, takes place as part of the process for paying vouchers 
submitted by placement contractors. Our findings suggest that there is 
reason to question whether this procedure is adequate to ensure that 
reported information is accurate. 



Labor has contracted with national labor and business organizations under 
sole source arrangements for more than 30 years. About a third of Job 
Corps’ vocational training is provided by such organizations contracted 
under sole source arrangements. Although Labor has failed to provide 
adequate support to justify sole source procurement for vocational 
training, it has nine sole source contracts with national labor and business 
organizations, totaling about $46 million (see table 4). 



Table 4: Labor s National Training 
Contractors 


Contractor 


Year of 
initial award 


Latest award 
(millions) 


Number of 
training slots 




AFL/CIO Appalachian Council 


1974 


$4.2 


542 




Home Builders Institute 


1974 


13.5 


4,090 




International Brotherhood of Painters 
and Allied Trades 


1969 


4.1 


1,280 




International Masonry Institute 


1971 


3.5 


910 




International Union of Operating 
Engineers 


1966 


2.5 


450 




National Plasterers and Cement 
Masons International Association 


1970 


5.3 


1,440 




T ransportation-Communication 
International Union 


1972 


4.2 


380 




United Brotherhood of Carpenters and 
Joiners of America 


1968 


6.3 


2,260 




United Auto Workers 


1978 


2.5 


396 




Total 




$46.1 


11,748 



Federal procurement regulations require several conditions to be met for 
an agency to award a noncompetitive contract. These include 
(1) establishing the need for services that can be provided by only one 
source, (2) documenting through a market survey or on some other basis 
that no other known entity can provide the required services, and 



Labor’s Justification 
for Sole Source 
Contracting Is 
Inadequate 
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(3) stating a plan of action the agency may take for removing barriers to 
competition in the future. 



Labor has offered three broad considerations in justifying its sole source 
awards rather than using competitive procedures in contracting with the 
national training contractors. The first is the contractors’ past relationship 
with Job Corps — that is, experience with Labor’s Employment and 
Training Administration, in general, and with Job Corps specifically and 
thorough knowledge of Job Corps’ procedures and operations. The second 
is organizational structure — that is, a large nationwide membership related 
to a trade and their strong relationship with national and local 
apprenticeship programs. The third is instructional capability — that is, a 
sufficiency of qualified and experienced instructors, the ability to provide 
training specifically developed for the learning level of Job Corps students, 
and the ability to recognize training as credit toward meeting the 
requirements of becoming a journey-level worker. In addition, Labor 
officials stated that a main reason it contracts on a sole source basis is that 
the contractors maintain an extensive nationwide placement network. 

With regard to Labor’s long-standing practice of awarding sole source 
contracts for a portion of Job Corps’ vocational training, our review of 
Labor’s current and proposed justification for its sole source contracts and 
our previous work on this issue raise questions about their use. 11 Labor’s 
sole source justification essentially lists the qualities Labor expects in a 
contractor. It does not establish that the services contracted for can be 
provided by only one source. Furthermore, Labor acknowledged that its 
national data system has no information to indicate the extent to which 
national training contractors are directly responsible for placing Job Corps 
participants in jobs. Labor’s proposed justification for upcoming contracts 
has many of the weaknesses of the current justification. 



Job Corps is an expensive job training program that provides 
comprehensive services to a severely disadvantaged population. For more 
than 30 years, Job Corps has been assisting young people who need and 
can benefit from an unusually intensive program, operated primarily in a 
residential setting. Labor and the Congress needi meaningful and accurate 
information if they are to effectively manage and oversee the Job Corps 
program. However, our work raises serious questions regarding Labor’s 
claims about Job Corps’ achievements. Labor’s reporting on the 
percentage of parti ' ts who are vocational completers includes many 




"GAO/HEHS-95-180, June 30, 1995. 



Page 19 



GAO/HEHS-99-15 Job Corps Vocational Training 



B-279218 



who have not actually completed their training; many have completed only 
one component of a vocational training program. Similarly, Labor’s 
reported statistics on the percentage of jobs obtained by participants that 
were related to the training they received are inaccurate. Reported job 
training matches include a significant number of jobs that have no 
apparent relationship to the training received and whose job titles have no 
apparent relationship to the employers’ business. 

In addition, Labor has continued its long-standing practice of awarding 
sole source contracts for a substantial portion of Job Corps’ vocational 
training — a practice we suggested it re-evaluate in 1995. To date, Labor has 
not provided adequate support to justify sole source procurement for 
vocational training services provided by the nine national labor and 
business organizations. Labor’s justification for sole source procurement 
does not explain or demonstrate the basis for Labor’s determination of 
need. 



Recommendations to Improvements are needed to ensure that the information used to assess 
,, q , f t u Job Cor P s Program performance is accurate and meaningful. Specifically, 

me becretary Ol Labor two of the measures used to judge the success of the Job Corps 

program — vocational completion and job training match — provide 
misleading information that overstates program outcomes. Therefore, we 
recommend that the Secretary of Labor 



• more accurately define and report information on the extent to which 
program participants complete vocational training and 

• develop a more accurate system of reporting training-related jobs and 
effectively monitor its implementation. 



In addition, because Labor has not presented adequate justification for its 
long-standing practice of contracting on a sole source basis with nine 
national labor and business organizations for vocational training, we 
recommend that the Secretary of Labor properly justify its use of 
noncompetitive procedures if it is to continue to award contracts for 
vocational training services. In so doing, the agency should assess whether 
vocational training could be served as well through contracts competed 
for locally or regionally. 



Agency Comments 



In comments on a draft of this report, Labor expressed concern about our 
conclusion that two performance measures — vocational training 
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completion and job training matches — overstated Job Corps’ success and 
misrepresented its accomplishments. Nevertheless, Labor agreed to 
implement our recommendations for improving the information provided 
by these two measures. 

Labor emphasized that it did not intend to overstate Job Corps program 
performance in any area. Labor further noted that it places strong 
emphasis on performance results and data integrity and is therefore 
concerned about the findings contained in the report. With regard to 
vocational training completion, Labor stated that it was never its intention 
that all students master all competencies on an occupation’s training 
achievement record. Instead, a set of competencies for each occupational 
area was developed by Labor, together with industry groups, to identify 
appropriate competency levels needed to qualify for particular 
occupations. For example, Labor noted that to qualify as a full mechanic 
would require completion of all competencies in the automotive area, but 
a participant could qualify as a mechanic’s helper or brake repair 
mechanic by completing a subset of the full automotive training 
achievement record. Labor also noted that even though vocational 
completion may be an imperfect measure, it is a good predictor of 
placement, job training match, and wages. However, Labor stated that it 
understood and shared our concern that the terminology used to report 
this information may be subject to misinterpretation. Therefore, Labor 
said that it would take immediate action to clarify the definition of 
vocational completion in all subsequent Job Corps publications. In 
addition, Labor noted that because of the perspective gained through the 
recent oversight hearings and our report, it would review the extent to 
which the current definition may provide insufficient incentive to some 
students to obtain the maximum amount of training within the vocational 
training program. Labor noted that in direct response to these issues, it has 
initiated a comprehensive and detailed analysis of vocational completion 
and stated that it will develop a more precise and comprehensive 
description of student completion levels. 

We believe that the actions Labor is taking to more clearly identify what it 
means by a vocational training completer will avoid future confusion 
about what is being reported. The actions will also clarify that it is not 
Labor’s intent to have all Job Corps participants complete all aspects of a 
vocational curriculum but, rather, to complete to a level that is appropriate 
for each individual. Such levels, as Labor noted, would correspond to 
industry-agreed competencies that would qualify a participant for a 
specific job. In addition, as Labor clarifies and refines its measures, it is 
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likely that more will be learned about the relationship between completing 
various levels of a vocational program and the degree of success a 
participant achieves. This is an important aspect of monitoring 
performance and could lead to program improvements. 

Regarding job training matches, Labor stated that it shares our concern 
about the validity of some of the matches identified in the report. Labor 
noted that it is currently changing to a different system for determining job 
training matches that will make the determination more manageable and 
easier to oversee. This new system is expected to be fully implemented by 
the close of this calendar year. In addition, Labor stated that it is 
developing more stringent quality control and oversight procedures to 
preclude questionable matches. 

We believe Labor’s proposed improvements to its assessment of whether 
job placements are related to the participants’ training and the monitoring 
of the reporting of these data will improve the validity and utility of this 
information. 

Regarding Labor’s use of sole source contracting with nine national labor 
unions and business organizations, Labor disagreed that it needed to do 
more to properly justify its use of noncompetitive procedures and 
expressed its belief that Job Corps’ training programs could not be served 
as well through locally or regionally competed procurements. Labor 
asserts that participants leaving national training contractor programs 
consistently achieve better outcomes, such as higher wages, than other 
participants. Labor also points out that it has received negligible responses 
to the last two invitations for interested organizations to submit capability 
statements for the administration and operation of vocational training 
programs and placement activities currently operated by national 
organizations. Labor contends that the continued strong performance of 
its sole source contracts and the lack of response to its attempts to solicit 
other qualified providers properly justify its decision to use 
noncompetitive procedures. Labor identified some changes, however, 
including that it will require the national contractors to report monthly on 
the number of participants who are placed directly into jobs and 
apprenticeships, and it has established higher performance standards for 
national training contractors. 

We continue to believe that Labor has not adequately justified its use of 
sole source contracts. Labor has been unable to determine the extent to 
which national training contractors are responsible for placing 
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participants and thus for their reported better performance. However, 
Labor’s new requirement for these contractors to report on their 
placements should improve Labor’s ability to assess their performance. 
From our review of Labor’s last two invitations for organizations to submit 
capability statements for the administration and operation of vocational 
training programs and placement activities, we conclude that the agency 
did not clearly state the goods and services required and was overly 
restrictive with respect to contractor qualifications. Thus, we believe that 
the two published invitations Labor cites were inadequate to inform 
potentially capable entities of an opportunity to compete or to afford them 
a reasonable opportunity to provide credible responses. As a result, Labor 
has not determined the availability of other potential sources and, 
therefore, has not properly justified its use of noncompetitive procedures. 

In addition, Labor suggested two points of technical clarification regarding 
the approval process that Job Corps centers use to change vocational 
offerings and the involvement of the national office in an employment 
initiative in one Job Corps region. We modified the report where 
appropriate. (Labor’s entire comments are printed in app. III.) 



We are sending copies of this report to the Secretary of Labor, the Director 
of the Office of Management and Budget, relevant congressional 
committees, and others who are interested. Copies will be made available 
to others on request. 

If you or your staff have any questions concerning this report, please call 
me at (202) 512-7014 or Sigurd R. Nilsen at (202) 512-7003. Major 
contributors to this report are listed in appendix IV. 

Sincerely yours, 




Carlotta C. Joyner 

Director, Education and Employment Issues 
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Appendix I 



Scope and Methodology 



We designed our study to collect information on the process the 
Department of Labor uses to obtain vocational training services for the 
Job Corps program and to identify efforts to ensure that vocational 
training is appropriate and relevant. We also sought to determine the 
extent to which program participants were completing their vocational 
training and obtaining jobs related to that training upon leaving the 
program. In doing our work, we interviewed Job Corps officials at the 
national and regional levels and conducted site visits at six judgmentally 
selected Job Corps centers — five established centers and one recently 
opened center. 

We augmented the information collected during the site visits with data 
from Labor’s Student Pay, Allotment, and Management Information 
System, a database containing nationwide information on all Job Corps 
participants. We analyzed information on Job Corps participants enrolled 
in program years 1995 and 1996, the two most recently completed program 
years for which data were available from this database. We did not, 
however, verify these data. 

We also administered a telephone survey to employers of a random sample 
of Job Corps participants who were reported as obtaining jobs that were 
related to the training they received in Job Corps but that we believed 
were questionable. We performed our work between January and 
July 1998 in accordance with generally accepted government auditing 
standards. 



Site Visits We visited six centers that used a variety of methods to provide vocational 

training and that served a majority of local participants — those who 
resided within 100 miles of a center. 



Table 1.1 : Job Corps Centers We 
Visited 



Labor region 


Job Corps center 


Location 


Contractor 


1 


Westover Job Corps Center 


Chicopee, Mass. The EC Corporation 


II 


Edison Job Corps Center 


Edison, N.J. 


Res-Care, Inc. 


IV 


Gulfport Job Corps Center 


Gulfport, Miss. 


Resource Consulting, Inc. 


IV 


Memphis Job Corps Center 


Memphis, Tenn. 


MINACT, Inc. 


VI 


David L. Carrasco Job 
Corps Center 


El Paso, Texas 


Texas Educational 
Foundation 


IX 


San Jose Job Corps Center 


San Jose, Calif. 


Career Systems 
Development Corporation 
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At these centers, we interviewed center directors and other relevant staff 
on vocational training activities, school-to-work efforts, and business and 
community relations. At five of the centers, we also reviewed participant 
data on vocational completion status and placement information. 12 Using 
the Dictionary of Occupational Titles and reported placement information, 
we compared the jobs participants reportedly obtained with the training 
they received to determine whether the jobs and training were related. We 
also contacted a random sample of 183 employers to clarify reported job 
information. 



Employer Survey 



At each of the five established centers, we reviewed placement 
information for every participant reported as having a job training match. 
The total number of reported job training matches ranged from 214 at one 
center to 369 at another center, for a total of 1,306. We classified each 
reported job training match into one of three categories: (1) good match, 
(2) questionable match, and (3) military. We considered all jobs listed in 
category 3 as being good matches. Using the Statistical Package for the 
Social Sciences sampling routine, we selected a simple random sample for 
each center of reported job training matches that we classified as 
questionable. The populations and sample sizes in this category for the five 
centers are shown in table 1.2. 



Table 1.2: Population and Sample Sizes 
by Center 


Job Corps center 


Reported job 
training matches 


Questionable 

matches 


Sample size 




David L. Carrasco 


214 


64 


■ 38 




Edison 


229 


115 


52 




Gulfport 


215 


138 


57 




San Jose 


279 


111 


54 




Westover 


369 


170 


61 




Total 


1,306 


598 


262 



During our survey, we asked employers to confirm the reported job title, 
identify the appropriate job title, if different, and describe the primary 
duties and responsibilities for the job for which the participant was hired. 
During the summer of 1998, we telephoned the employers for 262 reported 
job training matches. We made at least three attempts to contact each 
employer. After repeated calls, we were unable to reach or interview 79 
employers. We were able to reach and verify job information for the 183 



12 Because the Memphis Job Corps Center had recently opened, we did not review any information on 
participant outcomes. 
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other sample cases. Of these, 164 remained questionable after the 
discussion with the employers, and the remaining 19 were moved to the 
“good match” category. 

Given the results of our survey, we adjusted the number of questionable 
reported job training matches at each of the five centers as shown in table 
1.3. 



Table 1.3: Percentage of Questionable 
Job Training Matches at Five Centers 
Adjusted for Sample Results 







Adjusted number 


Percentage 


Job Corps center 


Reported job 


of questionable 


questionable 


training matches 


matches (sampling error %) 


David L. Carrasco 


214 


56 


26(3) 


Edison 


229 


106 


46 (3) 


Gulfport 


215 


134 


62(3) 


San Jose 


279 


98 


35 (4) 


Westover 


369 


147 


40(4) 


Total 


1,306 


541 


41 (2) 



All sample surveys are subject to sampling error— that is, the extent to 
which the results differ from what would be obtained if the whole 
population were administered the survey. Because the whole population 
was not surveyed, the true size of the sampling error cannot be known. 
However, it can be estimated from the responses to the survey. The 
estimate of sampling error depends largely on the number of respondents 
and the amount of variability in the responses. For this effort, center-level 
sampling errors were 3 to 4 percentage points with an overall sampling 
error of plus or minus 2 percentage points at the 95-percent confidence 
level. 



National and Regional 
Job Corps Offices 



We interviewed Labor officials at both the national and regional offices to 
obtain an overview of the vocational contracting process and information 
about initiatives to improve the relationship between Job Corps vocational 
training and employers’ needs. We also collected information on the 
contracting process, including information on the sole source 
procurement of national training contracts. 
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TSrtile 11.1: Reported and Full 
Completion Rates for Vocational 
Training Nationwide and at Five 
Centers for Program Year 1996 



Vocational training program 


Vocational completers 
completing only one 
component of training 


Auto repair 


23^ 


Building and apartment maintenance 


38 


Carpentry 


79 


Clerical 


67 


Electrical 


70 


Food service 


47 


Health occupations 


15 


Welding 


29 



Note: These are data for figure 1 . 



Table 11.2: Reported and Full 
Completion Rates for Vocational 




Percentage of participants 




Training Nationwide and at Five Job 
Corps Centers for Program Year 1996 


Center 


Vocational 

completers 


Full completers 


Partial completers 




National 


48% 14% 


34% 




Center A 


73 


27 


46 




Center B 


65 


11 


54 




Center C 


45 


20 


25 




Center D 


44 


22 


22 




Center E 


41 


13 


28 




Note: These are data for figure 2. 








Table 11.3: Reported and Questionable 
Training-Related Job Placement Rate 
at Five Job Corps Centers for Program 


Job Corps Center 


Jobs reported as Training-related jobs that 
training-related are questionable 


Year 1996 


A 




64% 


26% 




B 




69 


35 




C 




68 


40 




D 




55 


62 




E 




58 


46 



Note: These are data for figure 3. 
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Now on p. 11. 



U.S. Department of Labor Assistant Secretary for 

Employment and Training 
Washington, D.C. 20210 

SEP 2 5 1998 



Ms. Carlotta C. Joyner 

Director of Education and Employment Issues 
Health, Education and Human Services Division 
United States General Accounting Office 
441 G Street, NW, Room 5942 
Washington, DC 20548 

Dear Ms. Joyner: 

We appreciate the opportunity to provide comments in response to your draft report to the 
Chairman, Subcommittee on Human Resources, House Committee on Government Reform and 
Oversight, entitled “Job Corps: Links with Labor Market Improved But Vocational Training 
Performance Overstated.” In the response that follows, we address the major issues and 
recommendations identified in the draft report. 

We are particularly pleased that Job Corps’ efforts to increase the effectiveness of its vocational 
training through improved employer and community linkages are noted in your report. We are 
continuing these efforts and expanding them nationally, regionally, and locally. 

We are concerned with your finding in both the Results in Brief section (pages 3-4) of the draft 
report and the discussion which begins at the top of page 14 that the Job Corps program 
overstates program success for two performance measures: vocational completion and job 
training match. At the outset, let me emphasize that it has never been our intent to overstate Job 
Corps program performance in any area. We place strong emphasis on performance results, data 
integrity and credibility and therefore are concerned about the findings contained in the report. 
Job Corps is a results oriented program and we are continually seeking input and insight from 
others on how we can improve the program, the measures of performance, and the quality of the 
data. It is in this spirit that we are reviewing your findings. 

Vocational completion is one of nine measures of performance for which Job Corps centers are 
held accountable. I appreciate the opportunity to clarify Job Corps’ current definition of 
vocational completion, the context for use of that definition, as well as our plans for the future. 

In order to guide student instruction and assess student progress, Job Corps develops a Training 
Achievement Record (TAR), a list of skills for each major vocational program offered at a Job 
Corps center, such as health occupations, business and clerical, etc. Each vocational program 
offered at a Job Corps center has several different levels of completion, or step off levels, each 
of which has been determined by an industry advisory group to be suitable for at least entry level 
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employment, and each of which is designated on the TAR. Each student who achieves any one 
of those levels of completion in his/her vocational program is called a vocational completer. The 
only distinction currently made between levels of completion is between a basic completer and 
an advanced completer, the latter being those students who achieve all tasks listed on the TAR. 

Each TAR is developed with assistance from an Industry Advisory Group (IAG) comprised of 
employers and Job Corps practitioners. A TAR may include up to and over 100 skill 
competencies, all of which have been identified and are supported by the IAG. All TARs are 
comprised of a hierarchial order of occupational areas, each of which relates to a particular 
vocational field and is a discrete subset of the 100 or more competencies included in the TAR. 
The number of competencies required for mastery depends on the particular vocational field 
addressed by the TAR. For example, a student can be classified as a completer on the 
Automotive Training TAR if he or she masters the subset portion of competencies needed to 
qualify as a mechanic’s helper or brake repair or wheel mechanic. To qualify for these positions, 
the student does not have to master all the competencies listed in the TAR. These two 
occupational step-off levels offer very legitimate employment opportunities in the workplace. 

The advanced completer level, determined by mastery of all competencies on the TAR, is for 
qualification as a full mechanic to do engine diagnostics, a higher level employment opportunity. 
These three occupational areas, however, all require skill training and completion of each of 
these levels is considered an accomplishment for these students, which Job Corps refers to as 
vocational completers. 

In developing the TARS, it was never intended by Job Corps, or the employers on the IAG, that 
all competencies on a given TAR be mastered by every student in order to be considered a 
vocational completer who is employable in a given vocational field. In some instances, 
employers on a given IAG have indicated that it is not necessary to master all TAR competencies 
in order to obtain a good entry level job and have identified discrete job titles and associated skill 
requirements toward that end; in others, it is impractical for many students to remain in training 
long enough to master every skill reflected on the TAR because of immediate employment 
opportunities or extenuating circumstances such as pressure to return home, parental duties, and 
pressing financial needs. In addition, Job Corps’ identification of several completer levels within 
a particular vocational program enable Job Corps students to achieve the highest level within that 
vocational program that they can and consider themselves successful for that achievement. 

Job Corps developed vocational completion as one of its performance measures almost a decade 
ago, and while imperfect, it has served as a significant predicter of student placement rates, job 
training matches, and average wages. That is, students who are categorized as vocational 
completers have higher overall job placement rates, higher rates of job training match, and higher 
average wages. 
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However, notwithstanding this explanation of Job Corps’ current definition of vocational 
completion and the context in which it was designed to be used, we share the concerns of both 
the Subcommittee Chairman and the GAO that the terminology utilized to describe the 
attainment of vocational skills may be subject to misinterpretation. Since that was never our 
intent, we are assessing the full range of actions we will take in response to the report. We will 
take immediate action to clarify our definition in all subsequent Job Corps publications, 
including the forthcoming publication of Job Corps’ Program Year 1997 Annual Report. 

In addition, based on the perspective gained through the oversight hearing and this draft GAO 
report, we are reviewing the extent to which the use of the current definition of vocational 
completion may provide insufficient incentive to some students to obtain the maximum amount 
of training within that vocational program. In direct response to these issues, we have initiated 
a comprehensive and detailed analysis of vocational completion, including the time spent on 
center by students, levels of vocational completion achieved, and incentives provided to students 
to remain in vocational programs to attain maximum benefit. Where changes to existing TARs 
are warranted, we will take the appropriate corrective action. In addition, we will develop a more 
precise and comprehensive description of student completion levels, based on uniform criteria 
that will allow students to leave Job Corps prepared for viable career paths and long-term 
employment. We look forward to a continuing dialogue with your staff as we proceed with this 
effort. We expect this wQrk to be completed by July of 1999. We believe that these actions will 
result in meaningful changes to the current system of identifying the vocational skill 
achievements of students, and that they will fully address the recommendation to the Secretary, 
found on page 25 of the draft report, to “more accurately define and report information on the 
extent to which program participants complete vocational training ...” 

Regarding job training matches (JTMs) discussed on pages 18-22 of the draft report, we too 
share concerns about the validity of some of the matches identified in the report, and, as we had 
indicated in an earlier report, have already initiated changes to improve the accuracy of the data 
for this important performance indicator. Specifically, we are currently changing from a system 
that involves over 14,000 individual occupational codes, known as the Dictionary of 
Occupational Titles, to the Occupational Information Network (0*NET) System which has 
approximately 1,100 codes and is based on an analysis of the skills needed for a variety of jobs 
within broad categories of occupations. This will make the determination of job training matches 
more manageable and easier to oversee. The new system is expected to be fully implemented by 
the close of this calendar year. 

In addition, as part of the implementation of the new 0*NET system as the basis for the 
crosswalk between type of training and placement occupation, we are developing more stringent 
quality control and oversight procedures to preclude questionable matches. We look forward to 
working in collaboration with the GAO staff on these quality control efforts so we can more fully 
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benefit from the work your staff has already undertaken. We anticipate that these new controls 
will be in place by March of 1999. We believe that our decision to implement both the 0*NET 
System and quality control and oversight procedures responds directly to the recommendation to 
the Secretary, found on page 25 of the draft report, to “develop a more accurate system of 
reporting training-related jobs and effectively monitor its implementation.” 

Another major area of concern surrounds the issues raised in the justification for Job Corps' sole 
source procurement of vocational training with nine (9) labor and business organizations found 
on page 22. We believe that our investment with these national training contractors (NTCs) is a 
sound one that greatly benefits the students. These contracts take advantage of national networks 
available only through unions and homebuilder associations to provide access to both union and 
non-union jobs, including apprenticeships, in all labor markets. Students leaving NTC programs 
consistently achieve better outcomes than students leaving non-NTC programs. This is 
particularly true in the area of wages. Students placed in jobs out of NTC programs for the most 
recently completed program year (PY *97 — July l, 1997 through June 30, 1998) received an 
average hourly wage of $7.1 9 compared to $6.49 for those leaving non-NTC provided training, a 
difference of $.70 per hour. This difference increases to $.87 per hour when considering jobs 
related to training. Students leaving NTC programs who get jobs related to their training are 
placed at an average hourly wage of $7.55, compared to $6.68 for students leaving locally 
provided training. In our judgement, these differences are directly related to the effective, 
nationwide partnerships that NTCs have with local unions and organizations representing 
business and industry. 

In light of the above described performance comparisons, we believe that these types of training 
programs could not be served as well through locally or regionally competed procurements, as 
the draft report suggests on page 26. Nevertheless, as part of the Department of Labor’s sole 
source approval process, and in accordance with the Federal Acquisition Regulations (FAR), the 
Department published, in the Commerce Business Daily on February 17, 1998, an invitation for 
interested organizations to submit capability statements for the administration and operation of 
vocational training programs and placement activities currently operated by the NTCs. 

Interested parties were required to demonstrate their experience in coordinating, monitoring, 
overseeing and providing the necessary administrative support to effectively deliver these 
services on behalf of disadvantaged youth. This includes selection, assignment, training and 
supervision of instructors on a nationwide basis. No capability statements were received. 

On May 1 1 , 1998, the Department again published an expanded invitation in the Commerce 
Business Daily that included information specifically intended to generate substantial interest 
among qualified vocational training providers: such as total dollar value required for this effort, 
types of training and the number of centers that provide such training, and the total number of 
training slots to be delivered. In response to this second invitation, only two firms submitted 
capability statements, but the Department’s Procurement Review Board determined that neither 
firm had the capacity or experience to form the basis for competitive acquisition. Therefore, the 
Department decided to proceed with the NTCs on a sole source basis. We feel both the 
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continued strong performance of the NTCs and the negligible response to the Department’s 
repeated attempts to solicit qualified providers of vocational training services are in keeping with 
the recommendation, found on page 26 of the draft report, that the Secretary “...properly justify 
its use of noncompetitive procedures...” 

In addition to the above information, the following measures have been taken to further improve 
and document the performance of NTCs: 

(A) Beginning in program year 1998, performance standards for NTCs are higher than those 
established for other vocational training providers. In comparison to center operated programs, 
underperforming NTCs have a shorter period of time in which to improve or be dropped. 

(B) NTCs are required to provide follow up services, including placement, for up to a year after 
students have left their programs, and to report this information on a quarterly basis to the Job 
Corps National Office. An NTC Follow up Workgroup has been formed and meets periodically 
to review best practices and discuss methods and activities that can improve the delivery of 
follow up services. 

(C) Beginning with the contracts now being finalized, NTCs will be required to report each 
month on the number of students who are directly placed into jobs and apprenticeships after 
leaving the program. 

There are two points of technical clarification that we recommend, as follows: 


Now on p. 8. 


(1) On page 10 of the draft report, there is a reference to centers obtaining approval for changes 
in vocational training programs "... by completing the appropriate paperwork and submitting the 
request for change to the national office through the regional office.” Our current policy in this 
regard provides authority to the regional offices to approve center vocational changes that do not 
involve NTCs. Those changes that involve NTCs continue to require Job Corps national office 
approval. 


Now on p. 9. 


(2) On page 12 of the draft report, first paragraph, there is a statement that the region is 
sponsoring the employer involvement initiative and, further, that it was the region that selected 
the contractor. While we work in close partnership with the regional office on this initiative, it is 
actually sponsored by the Job Corps National Office, which also selected the contractor through 
the competitive process. 

We appreciate the opportunity to provide comments on the draft report, and in the areas of 
vocational completion and job training match. We are already using the perspective gained from 
your report and from the Chairman’s recent oversight hearing to carefully examine our current 
system and make improvements. Our commitment to program results based on accurate, 
credible data is strong and when we become aware of weaknesses or vulnerabilities, we are 
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committed to immediate action, which we have begun based on your draft report. The Job Corps 
staff and your staff already have a follow up meeting scheduled so we can get the maximum 
benefit from the work they have performed to date. 

Again, we also appreciate the acknowledgment of our efforts to strengthen and expand employer 
and community linkages. We trust that our responses to the issues and conclusions presented in 
this draft report will be taken into consideration as you develop the final report. If you have any 
questions, please contact Ms. Mary H. Silva, National Director of Job Corps, on 202-219-8550. 



Sincerely, 
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